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(54) Stereo range finder 

(57) The reference image data corresponding to 
one pixel block stored in a reference image line memory 
6 is output to a stereo matching circuit 8. The compara- 
tive image data corresponding to a "search range- 
stored in a comparative image line memory 7 is output 
to the stereo matching circuit 8 according to an instruc- 
tion of a search range address generator 1 3. The stereo 
matching circuit 8 calculates parallaxes by performing 



the stereo matching based on the reference image data 
and the comparative image data. A position where the 
search range S is set is corrected based on the amount 
of the deviation of an infinite distance corresponding 
point with respect to a position of the reference image 
data. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0001] The present invention relates to an appara- 
tus and a method for stereo matching and a method of 
calculating an infinite distance corresponding point. 
More particularly, the present invention relates to setting 
of a search range to be used in stereo matching. 

2. Description of the Related Art 

[0002] A stereo processing apparatus is known in 
the prior art, which obtains a pair of photographed 
images with a stereo camera system and calculates a 
distance to an object (or a parallax) shown in the photo- 
graphed images based on the pair of photographed 
images obtained from a stereo camera system. First, 
the processing apparatus identifies a correlated desti- 
nation of a given pixel block of one photographed image 
(right image, for example) in the other photographed 
image (left image, for example) (stereo matching). 
Then, the processing apparatus calculates a parallax of 
the pixel block, that is, the amount of deviation in a hor- 
izontal direction between the position of the pixel block 
(a correlated source) on the right image and the position 
of the correlated destination on the left image. 
[0003] As is commonly known, the correlated desti- 
nation of a given pixel block in the right image (reference 
image) exists on an epipolar line in the left image (com- 
parative image), or on the same horizontal line (at the 
same vertical coordinate) as the correlated source of 
the reference image is located, according to the princi- 
ple of a stereo method. Accordingly, the parallax is cal- 
culated as the amount of rightward deviation with 
respect to a horizontal coordinate of the correlated 
source on the reference image. Here, assuming that the 
distance from the stereo camera system to an object is 
infinitely large, its theoretical parallax is zero and the 
correlated destination exists at the same horizontal 
coordinate as the correlated source. A point of the hori- 
zontal coordinate on the epipolar line of the correlated 
destination calculated when the object is located at an 
infinite distance is referred to as an "infinite distance 
corresponding point' The amount of the rightward devi- 
ation (parallax) with respect to this infinite distance cor- 
responding point increases as the distance to the object 
decreases. Therefore, a search range to be used when 
performing stereo matching may be set to be a prede- 
termined range located to the right with respect to a 
point where the correlated destination is located most 
leftward, or the infinite distance corresponding point. 
[0004] A coordinate point of the correlated source 
on the reference image (hereinafter referred to as a "ref- 
erence coordinate point") and the infinite distance cor- 
responding point on the comparative image 



theoretically coincide with each other as seen above. In 
actuality, however, there are many cases in which they 
deviate from each other. The degree of suchdeviation is 
not uniform over the whole image but varies depending 

5 on the location in the image. A potential cause of this 
deviation is that focal distances of the right and left cam- 
eras differ from each other, causing one of the images 
to be enlarged or reduced compared to the other. The 
deviation can also occur due to distortion of camera 

10 lenses or an error in camera mounting position, for 
instance. As a consequence, there can arise such a sit- 
uation that the infinite distance corresponding point is 
located to the left (which is opposite to the normal direc- 
tion) of the reference coordinate point. Thus, there 

15 arises a problem that, if the reference coordinate point 
is set to as starting point of a search range, it becomes 
impossible to identify a correlated destination located to 
the left of this starting point. 

[0005] A potential solution to this problem would be 

20 to set the starting point of the search range to the left of 
the reference coordinate point with some margin. The 
amount of deviation of the infinite distance correspond- 
ing point from the reference coordinate point, however, 
is not uniform for different image areas as stated above. 

25 Therefore, when setting a margin of a fixed amount, it 
needs to be set to a larger amount than the amount of a 
maximum deviation of the infinite distance correspond- 
ing point. As a consequence, however, the search range 
widens as much as the increased amount of the margin 

30 thus provided, and this causes a problem that computa- 
tional complexity involved in stereo matching increases, 
resulting in a reduction in processing speed. In addition, 
the possibility of mismatching increases as much as the 
increase in the search range. Consequently, there also 

35 arises a problem that the reliability of calculated paral- 
laxes, or calculated distances, deteriorates. 

SUMMARY OF THE INVENTION 

40 [0006] The present invention has been made in 
view of the aforementioned problems. Accordingly, a 
first object of the present invention is to provide an 
apparatus and a method for stereo matching which can 
variably set a search range when performing stereo 

45 matching depending on the location on an image, and a 
method of calculating an infinite distance corresponding 
point 

[0007] A second object of the present invention is to 
avoid an expansion of a search range by setting an 
so appropriate search range depending on the location on 
an image, and thereby avoid an increase in computa- 
tional complexity required for the stereo matching oper- 
ation. 

[0008] A third object of the present invention is to 
55 reduce the possibility of mismatching by avoiding an 
expansion of the search range, and thereby prevent 
deterioration of the reliability of distance data. 
[0009] The objects can be achieved by a stereo 
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matching apparatus which comprises a stereo camera 
system, a memory, an address generator, and a stereo 
matching unit. The stereo camera system takes pictures 
of a predetermined area and producing first and second 
images of the predetermined area. The memory stores 5 
first image data within a reference pixel region in the first 
image and second image data on a horizontal line in the 
second image, the horizontal line being corresponding 
to a vertical position of the reference pixel region in the 
first image. The address generator sets a search range 10 
for a stereo matching and instructs to read out form the 
memory a part of the second image data which is within 
the search range and the first image data within the ref- 
erence pixel region. The stereo matching unit identifies 
a correlated destination of the reference pixel region by 75 
the stereo matching based on the part of the second 
image data which is within the search range and the first 
image data within the reference pixel region which are 
read out from the memory and calculates a parallax of 
the reference pixel region based on the correlated des- 20 
tination of the reference pixel region. The above-men- 
tioned address generator corrects a location of the 
search range for the reference pixel region based on the 
amount of deviation of an infinite distance correspond- 
ing point with respect to a horizontal position of the ref- 2 5 
erence pixel region. 

[0010] In the stereo matching apparatus of the 
invention, it is preferable that the address generator 
shifts the location of the search range in the same direc- 
tion as a direction of the deviation of the infinite distance 30 
corresponding point with respect to the horizontal posi- 
tion of the reference pixel region. The address genera- 
tor may shift a starting point of the search range in the 
same direction as the direction of the deviation of the 
infinite distance- corresponding point. Further, the 35 
address generator may shift an end point of the search 
range in the same direction as the direction of the devi- 
ation of the infinite distance corresponding point. 
[001 1] Further, in the stereo matching apparatus of 
the invention, it is preferable that the address generator 40 
identifies the amount of correction of the search range 
in relation to the amount of deviation of the infinite dis- 
tance corresponding point. 

[0012] Furthermore, it is also preferable that the 
stereo matching apparatus further comprises a correc- 45 
tion value memory. The correction value memory stores 
a correction value for correcting the location of the 
search range in relation to a location of the reference 
pixel region in the first image. In this case, the address 
generator sets the location of the search range for the so 
reference pixel region based on the location of the refer- 
ence pixel region and the correction value stored in the 
correction value memory for the location of the refer- 
ence pixel region. The address generator may set the 
starting point of the search range for the reference pixel 55 
region based on the location of the reference pixel 
region and the correction value stored in the correction 
value memory for the location of the reference pixel 
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region. 

[0013] In the above-mentioned stereo matching 
apparatus, the stereo camera system may comprise a 
pair of stereo cameras mounted on a vehicle, the pair of 
stereo cameras taking pictures of scenes outside of the 
vehicle. In this case, the stereo matching unit calculates 
a distance to an object outside the vehicle on the basis 
of the parallax of the reference pixel region. 
[0014] The above-mentioned objects can be also 
achieved by a stereo matching method comprising: tak- 
ing pictures of a predetermined area and producing first 
and second images of the predetermined area; identify- 
ing a reference pixel region in the first image; identifying 
a search range for the reference pixel region in the sec- 
ond image; correcting a location of the search range for 
the reference pixel region based on the amount of devi- 
ation of an infinite distance corresponding point with 
respect to a horizontal position of the reference pixel 
region; and identifying a correlated destination of the 
reference pixel region by searching through the search 
range. 

[0015] In the stereo matching method of the inven- 
tion, it is preferably advantageous that the step of cor- 
recting the location of the search range shifts the 
location of the search range in the same direction as the 
direction of the deviation of the infinite distance corre- 
sponding point with respect to the horizontal position of 
the reference pixel region. The step of correcting the 
location of the search range may shift a starting point of 
the search range in the same direction as the direction 
of the deviation of the infinite distance corresponding 
point with respect to the horizontal position of the refer- 
ence pixel region. Further, the step of correcting the 
location of the search range may shift an end point of 
the search range in the same direction as the direction 
of the deviation of the infinite distance corresponding 
point with respect to the horizontal position of the refer- 
ence pixel region. 

[0016] In the above-mentioned stereo matching 
method of invention, it is further advantageous that the 
step of correcting the location of the search range cor- 
rects the location of the search range with a correction 
value corresponding to the deviation of the infinite dis- 
tance corresponding point. 

[0017] Further, it is advantageous that the stereo 
matching method further comprises: calculating a dis- 
tance to an object outside a vehicle on the basis of the 
correlated destination of the reference pixel region. In 
this case, the step of taking pictures of the predeter- 
mined area takes pictures of scenes outside of the vehi- 
cle. 

[0018] Furthermore, it is advantageous that the 
stereo matching method further comprises: taking pic- 
ture of a screen by a pair of stereo cameras and produc- 
ing third and fourth images of the screen, the screen 
being drawn a first vertical line and a second vertical 
line at an interval equal to a baseline of the stereo cam- 
eras; identifying a deviation of a position of the second 
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vertical line shown on the fourth image with respect to a 
position of the first vertical line shown on the third 
image; and calculating an infinite distance correspond- 
ing point corresponding to each position on the first ver- 
tical line shown on the third image on the basis of the 5 
deviation of the position of the second vertical line. In 
this case, the step of correcting the location of the 
search range uses the infinite distance corresponding 
point thus calculated as the infinite distance corre- 
sponding point corresponding to the horizontal position w 
of the reference pixel region. 

[0019] Further, the above-mentioned objects can 
be achieved a stereo matching apparatus which com- 
prises a stereo camera system, a reference pixel region 
identifying unit, a search range identifying unit, a correc- 15 
tion unit and a correlated destination identifying unit 
The stereo camera system takes pictures of a predeter- 
mined area and producing first and second images of 
the predetermined area. The reference pixel region 
identifying unit identifies a reference pixel region in the 20 
first image. The search range identifying unit identifies a 
search range for the reference pixel region in the sec- 
ond image. The correction unit corrects a location of the 
search range for the reference pixel region based on the 
amount of deviation of an infinite distance correspond- 25 
ing point with respect to a horizontal position of the ref- 
erence pixel region. The correlated destination 
identifying unit identifies a correlated destination of the 
reference pixel region by searching through the search 
range. 30 
[0020] Furthermore, the above-mentioned objects 
can be achieved by a method of calculating an infinite 
distance corresponding point. The method comprising: 
taking picture of a screen by a pair of stereo cameras 
and producing first and second images of the screen, 35 
the screen being drawn a first vertical line and a second 
vertical line at an interval equal to a baseline of the 
stereo cameras; identifying a deviation of a position of 
the second vertical line shown on the second image 
with respect to a position of the first vertical line shown 40 
on the first image; and calculating an infinite distance 
corresponding point corresponding to each position on 
the first vertical line shown on the first image on the 
basis of the deviation of the position of the second verti- 
cal line. 45 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0021] 

50 

Fig. 1 is a block diagram of a stereo processing 
apparatus according to an embodiment of the 
present invention; 

Fig. 2 is a diagram showing a search range when a 
correction value is set to 0; 55 
Fig. 3 is a diagram showing a search range when a 
correction value is set to +4; 

Fig. 4 is a diagram showing a search range when a 



correction value is set to -4; 

Rg. 5 is a diagram for explaining a method of calcu- 
lating an infinite distance corresponding point 
according to the present invention; 
Rg. 6 is a diagram showing ideal photographed 
images; 

Rg. 7 is a diagram showing photographed images 
obtained in a case where a focal distance of a left 
camera is smaller than a focal distance of a right 
camera; 

Rg. 8 is a diagram showing photographed images 
obtained in a case where a focal distance of the left 
camera is smaller than a focal distance of the right 
camera and a lens of the left camera is greatly dis- 
torted; and 

Rg. 9 is a diagram showing a relationship between 
reference coordinate points and correction values 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

[0022] Rg. 1 is a block diagram of a stereo process- 
ing apparatus according to an embodiment of the 
present invention. A stereo camera system 1 which 
takes pictures of views outside a vehicle is mounted in 
the vicinity of its rear-view mirror. The stereo camera 
system 1 is composed of a pair of cameras 2a, 2b, each 
incorporating an image sensor such as a charge-cou- 
pled device (CCD). These cameras 2a, 2b are attached 
with a predetermined spacing in the direction of vehicle 
width therebetween. The main camera 2a for obtaining 
a reference image data is attached on the right side of 
the vehicle's forward direction. On the other hand, the 
sub-camera 2b for obtaining a comparative image is 
attached on the left side of the forward direction. 
[0023] In a condition in which the pair of the cam- 
eras 2a, 2b is synchronized, analog images output from 
the cameras 2a. 2b are respectively converted by ana- 
log-to-digital (A/D) converters 3a, 3b into digital images, 
each digital image having a predetermined brightness 
level (e.g., 256 gray scale levels) As a consequence, 
reference image data (brightness data) of 512 pixels in 
the horizontal direction by 200 pixels in the vertical 
direction is produced from an output signal of the main 
camera 2a. Also, from an output signal of the sub-cam- 
era 2b, comparative image data is obtained, the com- 
parative image having the same vertical length as the 
reference image and a greater horizontal length than 
the reference image (640 pixels in the horizontal direc- 
tion by 200 pixels in the vertical direction, for example). 
The digitized stereoscopic images (reference image 
and comparative image) for one frame (one view) are 
individually stored in frame memories 4a, 4b having a 
capacity to store digital data for one frame. 
[0024] The data stored in predetermined memory 
areas in the frame memories 4a, 4b are read out 
according to instructions from an address generator 12 
which constitutes part of an address generating circuit 
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10. The image data thus read out are transferred to an 
image correction circuit 5. The image correction circuit 5 
carries out such operations as a position correction and 
a brightness correction of the images so as to increase 
the accuracy of stereo matching which is performed by 5 
a stereo matching circuit 8 in a later stage. There are 
cases where the positions of the stereoscopic images 
are displaced due to a deviation of mounting positions 
(in a horizontal, vertical or rotational direction) of the 
right and left cameras 2a, 2b, a distortion of camera 
lenses, or an inclination of a light-sensing surface of 
each CCD, for instance. Accordingly, the deviation of the 
stereoscopic images is reduced by applying a linear 
shape correction like affine transformation or a nonlin- 
ear shape correction to the stereoscopic images. A non- 15 
linear correction method that can be used, for example, 
is a method described in Japanese Patent Application 
No. Hei. 10-132275(Japanese Patent Unexamined Pub- 
lication No. Hei. 1 1 -325889). The brightness correction 
is a correction for correcting differences in the sensitivity 20 
of the stereo processing apparatus. The brightness cor- 
rection, for example, Is a LOG correction for improving 
the contrast of low-brightness parts (described in Japa- 
nese Patent Application No. Hei. 1 1 -6879( Japanese 
Patent Unexamined Publication No. 2000-207555)), a 2 5 
shading correction for correcting brightness values 
based on a shading correction ratio stored for each pixel 
(described in Japanese Patent Application No. Hei. 5- 
58974(Japanese Patent Unexamined Publication No. 
Hei. 06-273171)), etc. 30 
[0025] The reference image data corrected by the 
image correction circuit 5 is stored in a memory area in 
a reference image line memory 6. More specifically, the 
reference image data of four horizontal lines in pixel 
units (one horizontal line in pixel block units 4x4) is 35 
stored. Also, the comparative image data corrected by 
the image correction circuit 5 is stored in a memory area 
in a comparative image line memory 7. More specifi- 
cally, the comparative image data of one horizontal line 
in pixel block units is stored. The expression "one hori- 40 
zontai line" used in the following explanation refers to 
one horizontal line in pixel block units. The address gen- 
erator 12 specifies the memory areas of the line memo- 
ries 6, 7 in which the image data are written. The 
reference image data output from the image correction 45 
circuit 5 is also entered to a microcomputer 9. 
[0026] The reference image data corresponding to 
one pixel block(reference pixel block) stored in the refer- 
ence image line memory 6 is read out according to an 
instruction from the address generator 12 and trans- so 
ferred to the stereo matching circuit 8. The data thus 
transferred is correlated source data to be used when 
performing stereo matching operation. In other words, it 
is pixel data of each pixel (brightness value) in the refer- 
ence pixel block(hereafter, referred to as "reference 55 
pixel block data") . Also, the comparative image data 
corresponding to a "search range" stored in the compar- 
ative image line memory 7 (hereafter referred to as 



"search range data") is read out according to an instruc- 
tion from a search range address generator 13 and 
transferred to the stereo matching circuit 8. The data 
thus transferred is pixel data of each pixel (brightness 
value) to be used in the stereo matching (hereafter, 
referred to as "search range data"). As the "search 
range 0 set by the search range address generator 1 3 is 
one of features of the present embodiment, it will be 
described later in detail. 

[0027] The stereo matching circuit 8 calculates par- 
allaxes by performing the stereo matching based on the 
reference pixel block data and the search range data. 
Since the parallax is calculated for each pixel block of 
4x4 pixels, a maximum of 128x50 parallaxes can be 
obtained from one complete frame of the reference 
image. Considering one pixel block (reference pixel 
block) in the reference image, a region (correlated des- 
tination) having a correlation with brightness properties 
of the one pixel block is identified by searching through 
the search range in the comparative image. While shift- 
ing a search point pixel by pixel along the aforemen- 
tioned epipolar line, the stereo matching circuit 8 
evaluates the state of correlation of each pixel block 
within the search range with the reference pixel block. 
[0028] The correlation between two pixel blocks can 
be evaluated by calculating city block distance CB 
shown in the following equation (1), for instance. In this 
equation, plij represents a brightness value of pixel (i, j) 
of one pixel block, while p2ij represents a brightness 
value of pixel (i, j) of the other pixel block. The city block 
distance CB is the sum (absolute value) of differences 
between the two geographically corresponding bright- 
ness values plij, p2ij for one complete pixel block, 
wherein the smaller the differences, the greater the cor- 
relation between the two pixel blocks. 

CB= Zlplij -p2ijl (1) 

[0029] Basically, a pixel block which gives a mini- 
mum value of city block distance CB among city block 
distances CB calculated for the each pixel block existing 
on the epipolar line is judged to be the correlated desti- 
nation of the reference pixel block. The amount of devi- 
ation between the correlated destination thus identified 
and the reference pixel block is the parallax. 
[0030] The stereo matching circuit 8 also evaluates 
the presence or absence of a brightness edge (the 
amount of brightness change) in the horizontal direction 
of the pixel block, as well as its relationship with maxi- 
mum and minimum values of the city block distance CB 
calculated on the same epipolar line. Furthermore, to 
ensure reliability as distance data, the stereo matching 
circuit 8 performs a filtering operation on the parallax 
and outputs only such parallax that has been judged to 
be valid (the parallax which has been judged to be valid 
is hereinafter referred to simply as the "parallax") . Ref- 
erence should be made to Japanese Patent Unexam- 
ined Publication No. 5-114099 for details of hardware 
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configuration and filtering operation for calculating the 
city block distance, if necessary, since they are dis- 
closed therein. The parallax calculated through the 
aforementioned operation is transferred to the micro- 
computer 9. 5 
[0031] In this embodiment, the address generating 
circuit 10 has a function of setting a search range S to 
be used when performing the stereo matching. To real- 
ize this function, the address generating circuit 10 has a 
reference pixel block position designator 11 , a correc- 10 
tion value memory 14 and the search range address 
generator 13. The position where the search range S is 
set is corrected as follows according to the status of an 
infinite distance corresponding point x of the position 
(ip, jp) of a reference pixel block PB. 15 
[0032] First, for the reference pixel block PB to be 
subjected to the stereo matching, the reference pixel 
block position designator 1 1 identifies its position (ip, jp) 
on the reference image (where 0<ip<127, 0<jp<49) in 
pixel block units. Then, the position (ip, jp) of the refer- 20 
ence pixel block PB is output to the search range 
address generator 13 and the correction value memory 
14. 

[0033] When the position (ip, jp) of the reference 
pixel block PB has been identified by the reference pixel 25 
block position designator 1 1 , reference is made to a 
search starting point correction table 15 stored in the 
correction value memory 14, and then a correction 
value PS for the reference pixel block PB is identified. 
The correction value memory 14 is formed of an electri- 30 
cally erasable programmable read-only memory (EEP- 
ROM), for example. This correction value PS is a value 
for correcting a starting point of the search range S in 
the comparative image, or for correcting the search 
starting point, applicable when searching for a corre- 35 
lated destination of the reference pixel block PB. Cor- 
rection values PS are set and written in the correction 
value memory 14, based on deviation characteristics of 
an infinite distance corresponding point % on a photo- 
graphed image measured at the time of a system initial- 40 
ization (i.e., in an adjustment process in a factory or 
during readjustment at a dealer). The infinite distance 
corresponding point % and the correction value PS gen- 
erally have the following relationship: 
(Relationship between infinite distance corresponding 45 
point % and correction value PS) 

1 . The correction value PS is of the same sign as 
the direction (plus or minus) of deviation of the infi- 
nite distance corresponding point % with respect to so 
a reference coordinate point X (the coordinate point 

of the reference pixel block PB). 

2. The greater the amount of deviation of the infinite 
distance corresponding point x with respect to the 
reference coordinate point X, the greater the 55 
amount of correction that the correction value PS 
indicates. 
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[0034] When the correction value PS correspond- 
ing to the position (ip, jp) of the reference pixel block has 
been read out from the correction value memory 14, a 
starting point SAD of a search range S and an end point 
EAD thereof are calculated based on the following 
equations by the search range address generator 13 
and the search range S is set: 
(Setting of search range S) 

Starting point SAD = (4 x ip - p) + PS 

End point EAD = SAD + 131 

[0035] Specifically, the reference coordinate point X 
(in pixel units) is calculated by multiplying the horizontal 
coordinate ip of the reference pixel block PB by the 
number of horizontally arranged pixels 4 in one pixel 
block. A basic starting point A is calculated by subtract- 
ing a specified margin p from the reference coordinate 
point X (or by shifting the reference coordinate point X 
leftward). The starting point SAD of the search range S 
is a value obtained by adding the correction value PS to 
the basic starting point A. Also, the end point EAD of the 
search range S is a value obtained by adding a speci- 
fied value 131 to the starting point SAD, so that this 
value is uniquely determined when the starting point 
SAD is determined. The search range S is therefore the 
range (as wide as 132 pixels) from the starting point 
SAD to the end point EAD. 

[0036] The search range address generator 13 
gives an instruction to the comparative image line mem- 
ory 7 that the comparative image data (search range 
data) within the search range S thus set should be read 
out. The search range data read out is transferred to the 
stereo matching circuit 8. The stereo matching circuit 8 
then calculates the aforementioned city block distance 
CB based on the reference pixel block data transferred 
from the reference image line memory 6 and the search 
range data. 

[0037] Fig. 2 is a diagram showing a search range S 
set when a correction value PS is 0. The correction 
value PS is set according to its relationship with the 
measured infinite distance corresponding point x as 
described above. Thus, the correction value PS of 0 
means that the deviation of the infinite distance corre- 
sponding point x with respect to the reference coordi- 
nate point X is 0 or, in other words, the infinite distance 
corresponding point x exists at the same horizontal 
coordinate as the reference coordinate point X. There- 
fore, even when search operation is begun from the 
basic starting point A (correction value PS = 0), the infi- 
nite distance corresponding point x wi|1 in no case 
depart from the search range S. The end point EAD of 
the search range S in this case is A + 1 31 . 
[0038] Fig. 3 is a diagram showing a search range S 
set when a correction value PS is +4. Since the correc- 
tion value PS is +4, the starting point SAD is set to a 
value obtained by adding the correction value 4 to the 
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basic starting point A and the end point EAD is set to 
+1 35. The correction amount PS of +4 indicates that the 
infinite distance corresponding point % deviates right- 
ward by the amount of correction equal to 4 with respect 
to the reference coordinate point X. Therefore, even 
when the starting point SAD is shifted rigfitward by the 
amount of correction, the'infinite distance correspond- 
ing point % will in no case depart from the search range 
S, and it is assured that the infinite distance correspond- 
ing point x exists in the right-hand vicinity of the starting 
point SAD. 

[0039] Fig. 4 is a diagram showing a search range S 
set when a correction value PS is -4. Since the correc- 
tion value PS is -4, the starting point SAD Is set to a 
value obtained by subtracting the correction value 4 
from the basic starting point A and the end point EAD is 
set to +127. The correction amount PS of -4 indicates 
that the infinite distance corresponding point % deviates 
leftward by the amount of correction equal to 4 with 
respect to the reference coordinate point X. Therefore, it 
is possible to prevent the infinite distance corresponding 
point % ^om departing from the search range S by shift- 
ing the starting point SAD leftward as much as the 
amount of correction. It is assured that the infinite dis- 
tance corresponding point x exists in the right-hand 
vicinity of the starting point SAD in this case as well. 
[0040] Considering that the deviation of the infinite 
distance corresponding point x with respect to the refer- 
ence coordinate point X is not uniform for different 
image areas, the starting point SAD and the end point 
EAD of the search range S are shifted in this embodi- 
ment. In contrast, if the search range S is set at a fixed 
location, it is necessary to shift the starting point of the 
search range S leftward by a margin determined in con- 
; sideration of the amount of a maximum deviation of the 
infinite distance corresponding point x and also shift its 
end point rightward by a relatively large margin. If the 
location of the search range is made variable according 
to the deviation of the infinite distance corresponding 
point as in the present embodiment, it is not necessary 
to set such large margins so that an expansion of the 
search range will not result. As a consequence, it is pos- 
sible to avoid an increase in computational complexity 
required for the stereo matching and to prevent a reduc- 
tion in processing speed. It is also possible to reduce 
the possibility of mismatching in the stereo matching, 
because it is not necessary to set a large search range. 
It is therefore possible to ensure the accuracy of calcu- 
lated parallaxes, or calculated distances. 
[0041] In the above-mentioned embodiments, the 
starting point SAD of the search range S is obtained by 
adding the correction value PS to the basic starting 
point A, and the end point EAD thereof is obtained by 
adding the specified value 1 31 to the starting point SAD. 
However, the ending point EAD of the search range S 
may be obtained by adding the correction value PS. In 
this case, the starting point SAD is obtained by subtract- 
ing the specified value 131 to the ending point EAD. The 



deviation of the infinite distance corresponding point x 
with respect to the reference coordinate point X can be 
obtained by the following method. Fig. 5 is a diagram for 
explaining a method of calculating the infinite distance 

5 corresponding point % according to the present inven- 
tion. First, to identify the infinite distance corresponding 
_^^ v point Xf a special screen 21 is placed at a predeter- 
mined position in front of the stereo camera system 1 . 
Six vertical lines are drawn on this screen 21, wherein 

to an interval between the adjacent vertical lines is made 
equal to a length of the baseline of the stereo camera 
system 1, or to an interval between the right camera 2a 
and the left camera 2b. The individual vertical lines are 
numbered to permit their identification. Specifically, the 

15 rightmost vertical line on the screen is numbered "com- 
parison No. 1 ," from where the rest of the vertical lines 
are sequentially numbered "comparison No. 2" to "com- 
parison No. 5." Also, the second vertical line from the 
right side of the screen is numbered "reference No. 1 ," 

20 from where the rest of the vertical lines are sequentially 
numbered "reference No. 2" to "reference No. 5." The 
screen 21 is positioned in such a way that the vertical 
line "reference No. 3" would be shown at the center of a 
photographed image (reference image) when the 

25 screen 21 is photographed by the right camera 2a. As a 
result, the vertical line "comparison No. 3" is shown at 
the center of a photographed image (comparative 
image) taken by the left camera 2b. 
[0042] When such screen 21 is photographed with 

30 the stereo camera system 1, photographed images as 
shown in Fig. 6 are obtained under an ideal condition. 
Under this condition, the positions of the vertical lines 
"comparison No. 1 " to "comparison No. 5" shown on the 
comparative image perfectly coincide with the positions 

35 of the corresponding vertical lines "reference No. 1" to 
"reference No. 5" shown on the reference image. The 
interval between the individual vertical lines is made 
equal to the length of the baseline of the stereo camera 
system 1 . Therefore, according to characteristics of the 

40 stereo method, the corresponding vertical lines "com- 
parison No. 1 " to "comparison No. 5" on the compara- 
tive image correspond to infinite distance corresponding 
points x at the respective positions (their deviation is 0 
in this case). 

45 [0043] If there is any influence of variations in focal 
distances or lens distortion of the right and left cameras 
2a, 2b, or an error in horizontal positions of right and left 
cameras 2a, 2b, the photographed images as shown in 
Fig. 6 would not be obtained. Fig. 7 is a diagram show- 
so ing photographed images obtained in a case where the 
focal distance of the left camera 2b is smaller than the 
focal distance of the right camera 2a (where the left 
camera 2b is more wide-angled). In this case, the com- 
parative image shown will be reduced In the horizontal 
55 direction, centering on the center of the image. There- 
fore, the vertical lines "comparison No. 1 " and "compar- 
ison No. 2" are shifted rightward compared to those 
under the ideal condition, while the vertical lines "com- 
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parison No. 4" and "comparison No. 5" are shifted left- 
ward compared to those under the ideal condition. For 
this reason, an object (which has a finite distance) 
shown on the individual vertical lines 'reference No. 1 " 
to "reference No. 5" is always located to the right of the 5 
positions of their corresponding vertical lines "compari- 
son No. 1" to "comparison No. 5" (i.e., the infinite dis- 
tance corresponding points x). 

[0044] Fig. 8 is a diagram showing photographed 
images obtained in a case where the focal distance of 
the left camera 2b is smaller than the focal distance of 
the right camera 2a and the lens of the left camera 2b is 
greatly distorted. Since the aforementioned factors 
often exist in combination in actuality, there are many 
cases where the photographed images shown in Fig. 8 
are obtained. Unlike the case of Fig. 7, the deviation of 
the corresponding vertical lines "comparison No. 1 " to 
"comparison No. 5" varies depending not only on hori- 
zontal location but also on vertical location in the 
present case. Namely, the deviation of the infinite dis- 
tance corresponding points % varies depending on the 
vertical location in the image areas. 
[0045] The deviation of the infinite distance corre- 
sponding point x for each location in the image areas is 
determined and the correction value PS corresponding 
to the deviation is individually set for each location as 
seen above. Fig. 9 is a diagram showing a relationship 
between reference coordinate points (coordinate points 
of reference pixel blocks) and correction values PS. 
When the infinite distance corresponding points % coin- 
cide with their corresponding reference coordinate 
points X over the whole of horizontal coordinates on the 
images as illustrated in Fig. 6, the correction value PS is 
set to 0 as shown by a symbol "a" in Fig. 9. In contrast, 
when the vertical lines "reference No. 1 ," "reference No. 
2," "reference No. 4" and "reference No. 5" are shifted 
toward the central vertical line "reference No. 3" as illus- 
trated in Fig. 7, the correction value PS is set in such a 
way that it linearly decreases with an increase in hori- 
zontal coordinate value as shown by a symbol "b" in Fig. 
9. Also, under conditions depicted in Fig. 8, correction 
values PS are set in such a way that it nonlinearly 
decrease with an increase in horizontal coordinate 
value as shown by a symbol "c" in Fig. 9. A curve "c" of 
Fig. 9 shows the correction values PS for the same ver- 
tical coordinate. Furthermore, when there exists a devi- 
ation in a yaw direction between the right and left 
cameras 2a, 2b, characteristics shown in Fig. 9 shift in a 
vertical direction. 

[0046] It is possible to determine the deviation of 
the infinite distance corresponding points x with respect 
to the reference coordinate points by measuring the 
deviation of corresponding vertical lines "comparison 
No. 1 " to "comparison No. 5" to vertical lines "reference 
No. 1 " to "reference No. 5" as seen above. As thus far 
described, the correlated destination of a given refer- 
ence pixel block PB always exists to the right of the cor- 
responding vertical line "comparison No. 1" which 



corresponds to the infinite distance corresponding point 
X- Therefore, the correction value PS should be set to a 
search point SAD which always includes this infinite dis- 
tance corresponding point %. 

[0047] It is possible to set an appropriate search 
range for each reference pixel block by correcting the 
location of the search range according to the deviation 
of an infinite distance corresponding point with respect 
to a reference coordinate point of the reference pixel 
block as seen above in this invention. Therefore, it is not 
necessary to set a large margin for the search range 
considering a maximum deviation of the infinite distance 
corresponding point as described above. Since an 
expansion of the search range can be avoided as a con- 
sequence, it is possible to avoid an increase in compu- 
tational complexity required for stereo matching. 
Furthermore, since the possibility of mismatching can 
be reduced by avoiding the expansion of the search 
range, it is possible to prevent deterioration of the relia- 
bility of distance data. 

[0048] While the presently preferred embodiment of 
the present invention has been shown and described, it 
is to be understood that this disclosure is for the pur- 
pose of illustration and that various changes and modi- 
fications may be made without departing from the scope 
of the invention as set forth in the appended claims. 

Claims 
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20 
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30 1. A stereo matching apparatus comprising: 

a stereo camera system for taking pictures of a 
predetermined area and producing first and 
second images of the predetermined area; 
35 a memory for storing first image data within a 

reference pixel region in the first image and 
second image data on a horizontal line in the 
second image, the horizontai line being corre- 
sponding to a vertical position of the reference 
40 pixel region in the first image: 

an address generator for setting a search 
range for a stereo matching and instructing to 
read out from said memory a part of the sec- 
ond image data which is within the search 
45 range and the first image data within the refer- 

ence pixel region; and 

a stereo matching unit for identifying a corre- 
lated destination of the reference pixel region 
by the stereo matching based on the part of the 
so second image data which is within the search 

range and the first image data within the refer- 
ence pixel region which are read out from said 
memory and calculating a parallax of the refer- 
ence pixel region based on the correlated des- 
55 tination of the reference pixel region, 

wherein said address generator corrects a 
location of the search range for the reference 
pixel region based on the amount of deviation 
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of an infinite distance corresponding point with 
respect to a horizontal position of the reference 
pixel region. 

2. The stereo matching apparatus as recited in claim 5 

1, wherein said address generator shifts the loca- 
tion of the search range in the same direction as a 
direction of the deviation of the infinite distance cor- 
responding point with respect to the horizontal posi- 
tion of the reference pixel region. io 

3. The stereo matching apparatus as recited in claim 

2, wherein said address generator shifts a starting 
point of the search range in the same direction as 

the direction of the deviation of the infinite distance is 
corresponding point. 

4. The stereo matching apparatus as recited in claim 
2, wherein said address generator shifts an end 
point of the search range in the same direction as 20 
the direction of the deviation of the infinite distance 
corresponding point. 

5. The stereo matching apparatus as recited in one of 
claims 1 to 4, wherein said address generator iden- 25 
tifies the amount of correction of the search range 

in relation to the amount of deviation of the infinite 
distance corresponding point. 

6. The stereo matching apparatus as recited in claim 30 

2, further comprising: 

a correction value memory for storing a correc- 
tion value for correcting the location of the 
search range in relation to a location of the ref- 35 
erence pixel region in the first image; 
wherein said address generator sets the loca- 
tion of the search range for the reference pixel 
region based on the location of the reference 
pixel region and the correction value stored in 40 
said correction value memory for said location 
of the reference pixel region. 

7. The stereo matching apparatus as recited in claim 

3, further comprising: 45 

a correction value memory for storing a correc- 
tion value for correcting the location of the 
search range in relation to a location of the ref- 
erence pixel region in the first image; so 
wherein said address generator sets the start- 
ing point of the search range for the reference 
pixel region based on the location of the refer- 
ence pixel region and the correction value 
stored in said correction value memory for said 55 
location of the reference pixel region. 

8. The stereo matching apparatus as recited in claim 



1, wherein said stereo camera system comprises a 
pair of stereo cameras mounted on a vehicle, the 
pair of stereo cameras taking pictures of scenes 
outside of the vehicle, and said stereo matching 
unit calculates a distance to an object outside the 
vehicle on the basis of the parallax of the reference 
pixel region. 

9. A stereo matching method comprising: 

taking pictures of a predetermined area and 
producing first and second images of the pre- 
determined area; 

identifying a reference pixel region in the first 
image; 

identifying a search range for the reference 
pixel region in the second image; 
correcting a location of the search range for the 
reference pixel region based on the amount of 
deviation of an infinite distance corresponding 
point with respect to a horizontal position of the 
reference pixel region; and 
identifying a correlated destination of the refer- 
ence pixel region by searching through the 
search range. 

10. The stereo matching method as recited in claim 9, 
wherein said step of correcting the location of the 
search range shifts the location of the search range 
in the same direction as the direction of the devia- 
tion of the infinite distance corresponding point with 
respect to the horizontal position of the reference 
pixel region. 

11. The stereo matching method as recited in claim 10, 
wherein said step of correcting the location of the 
search range shifts a starting point of the search 
range in the. same direction as the direction of the 
deviation of the infinite distance corresponding 
point with respect to the horizontal position of the 
reference pixel region. 

12. The stereo matching method as recited in claim 10, 
wherein said step of correcting the location of the 
search range shifts an end point of the search 
range in the same direction as the direction of the 
deviation of the infinite distance corresponding 
point with respect to the horizontal position of the 
reference pixel region. 

13. The stereo matching method as recited in one of 
claims 9 to 12, wherein said step of correcting the 
location of the search range corrects the location of 
the search range with a correction value corre- 
sponding to the deviation of the infinite distance 
corresponding point. 

14. The stereo matching method as recited in claim 9, 
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further comprising: 

calculating a distance to an object outside a 
vehicle on the basis of the correlated destina- 
tion of the reference pixel region, 5 
wherein said step of taking pictures of the pre- 
determined area takes pictures of scenes put- 
side of the vehicle. 

15. The stereo matching method as recited in claim 9, 10 
further comprising: 

taking picture of a screen by a pair of stereo 
cameras and producing third and fourth images 
of the screen, said screen being drawn a first is 
vertical line and a second vertical line at an 
interval equal to a baseline of the stereo cam- 
eras; 

identifying a deviation of a position of the sec- 
ond vertical line shown on the fourth image 20 
with respect to a position of the first vertical line 
shown on the third image; and 
calculating an infinite distance corresponding 
point corresponding to each position on the first 
vertical line shown on the third image on the 25 
basis of the deviation of the position of the sec- 
ond vertical line, 

wherein said step of correcting the location of 
the search range uses the infinite distance cor- 
responding point thus calculated as the infinite 30 
distance corresponding point corresponding to 
the horizontal position of the reference pixel 
region. 

16. A method of calculating an infinite distance corre- 35 
sponding point, the method comprising: 

taking picture of a screen by a pair of stereo 
cameras and producing first and second 
images of the screen, said screen being drawn 40 
a first vertical line and a second vertical line at 
an interval equal to a baseline of the stereo 
cameras; 

identifying a deviation of a position of the sec- 
ond vertical line shown on the second image as 
with respect to a position of the first vertical line 
shown on the first image; and 
calculating an infinite distance corresponding 
point corresponding to each position on the first 
vertical line shown on the first image on the so 
basis of the deviation of the position of the sec- 
ond vertical line. 

17. A stereo matching apparatus comprising: 

ss 

a stereo camera system for taking pictures of a 
predetermined area and producing first and 
second images of the predetermined area; 



a reference pixel region identifying unit for iden- 
tifying a reference pixel region in the first 
image; 

a search range identifying unit for identifying a 
search range for the reference pixel region in 
the second image; 

a correction unit for correcting a location of the 
search range for the reference pixel region 
based on the amount of deviation of an infinite 
distance corresponding point with respect to a 
horizontal position of the reference pixel region; 
and 

a correlated destination identifying unit for 
identifying a correlated destination of the refer- 
ence pixel region by searching through the 
search range. 
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(54) Stereo range finder 



(57) The reference image data corresponding to 
one pixel block stored in a reference image line memory 
6 is output to a stereo matching circuit 8. The compar- 
ative image data corresponding to a "search range" 
stored in a comparative image line memory 7 is output 
to the stereo matching circuit 8 according to an instruc- 
tion of a search range address generator 1 3. The stereo 



matching circuit 8 calculates parallaxes by performing 
the stereo matching based on the reference image data 
and the comparative image data. A position where the 
search range S is set is corrected based on the amount 
of the deviation of an infinite distance corresponding 
point with respect to a position of the reference image 
data. 
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